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Abstract 

This study sought to understand and improve effectiveness 
in institutional research (IR) by interviewing, observing, and 
analyzing resumes of IR practitioners who have been identified 
by their colleagues as particularly effective in having an 
impact on decision-making, planning, and policy formation. 
Ten themes for effectiveness in IR were identified, including 
understanding your institution, multiple perspectives of its 
constituents, and academe in general; engagement, visibility, 
relationships, and trust; helping leaders make meaning of 
IR findings and use of IR work in the institution; viewing IR 
as a means to a greater goal; professional colleagues and 
professional development; workload, time management, 
and tools; being self-reflective and proactive; attention to 
detail and quality control; personal and professional qualities; 
and technical competency. A model of the components of 
effectiveness in IR is provided. 



William E. Knight 

Bowling Green State University 



In Their Own Words: Effectiveness in 
Institutional Research 

Institutional research (IR) is viewed by a variety of 
constituencies as essential to allowing higher education to 
survive and thrive in the current environment. The funding crisis, 
competition from both traditional and nontraditional sectors, 
pressures to show effectiveness in student learning, contributions 
to economic development and community engagement, and the 
need for effective enrollment management are just a few examples 
of areas where IR has been called upon to contribute to decision- 
making and planning (Howard, 2001 ; Hutchings & Shulman, 2006; 
Kuh & Associates, 2005; Saupe, 1 990). 

As Peterson (1 999) points out, the profession is fortunate in 
that it has a long history of self-reflection. Authors have pondered 
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topics such as what IR is (Dressel & Associates, 1 971 ; 
Fincher, 1 985; Lasher & Firnberg, 1 983; Peterson 
& Corcoran, 1 985; Saupe, 1 990), how it should be 
organized (Presley, 1 990), what skills and expertise 
it requires (Suslow, 1 972; Terenzini, 1 993), and what 
roles and activities practitioners should embrace 
(Billups & DeLucia, 1 990; Chan, 1 993; Chase, 1 979; 
Gubasta, 1 976; Hurst, Matier, & Sidle, 1 998; Keller, 
1995; Lohmann, 1998; Matier, Sidle, & Hurst, 1995; 
Sanford, 1983, 1995; Terenzini, 1995; Volkwein, 1990, 
1999). Surveys (e.g., Lindquist, 1999; Muffo, 1999) 
have described the characteristics, settings, and 
activities of institutional researchers. Chambers and 
Gerek (2007) ask the questions "Are we doing things 
the right way?" and "Are we doing the right things?" 
Those authors also studied the Association for 
Institutional Research (AIR) Code of Ethics (AIR, 2001) 
and posited 

If the Code of Ethics is used as the 
foundation, institutional research can 
include any [activity] in which we have a 
competency, in which we have competent 
friends, or in which we can "acquire the 
necessary competency prior to doing the 
research. (Chambers and Gerek, 2007, p. 1) 
Numerous professional development opportunities, 
including conferences, workshops, institutes, 
publications, graduate coursework and certification, 
grant programs, and professional organizations, 
exist to allow us to maintain and enhance our career 
effectiveness (Knight, 2003). 

Some institutional researchers have studied the 
characteristics and experiences of their colleagues 
to learn how they can enhance effectiveness in 
their roles at their institutions or organizations. 
Augustine (2001 ) concluded that effective use 
of IR studies is associated with transmission of 
findings through multiple media, congruence in 
disciplinary backgrounds between the researcher 
and decision-makers, organizational placement 
of the IR office, frequent communication between 
researchers and decision-makers, use of qualitative 
methods, and provision of advice on use of 
research results. Clyburn (1 991 ) found that many 
small, private colleges lacked an IR function and, 
where it did exist, it tended to suffer from lack of 



coordination, commitment, and support. Delaney's 
(1 997) survey of institutional researchers at New 
England colleges and universities revealed that the 
scope of the function, the reporting relationship, 
and the size and qualifications of the staff varied 
significantly with institutional size, level, and 
control. She also found that the likelihood 
of involvement of IR offices with research (as 
contrasted with reporting), planning, and policy 
development varied with institutional size, level, 
control, and staff size and qualifications. Delaney 
(2000) concluded that institutional researchers 
who perceived themselves to be more effective felt 
that they had more opportunities for autonomy 
and leadership and were more likely to have their 
work used in executive decision-making, include 
policy recommendations in reports, conduct follow- 
up studies on the impact of their work, have a 
doctorate, be part of a strong professional network, 
and describe their positions as challenging. 

Using a survey of institutional researchers in the 
Northeast, Delaney (2001) identified workload, 
limited opportunity for advancement, stress, lack 
of recognition, concern for producing quality work 
within time constraints, and the lack of financial 
and moral support as the most common challenges 
practitioners face to their engagement in policy. 

She concluded through the use of a path analysis 
model that practitioners who were in higher 
positions and who had more experience and higher 
education levels, a mentor, a strong professional 
network, and an independent job structure can 
more effectively meet such challenges and actively 
engage in policy development. Huntington and 
Clagett (1 991 ) learned that the most prevalent 
problems experienced by institutional researchers 
include workload and staffing, perceptions of 
the function, access to institutional leaders, and 
access to and reliability of institutional information 
systems. Knight, Moore, and Coperthwaite (1 997) 
sought to empirically validate Terenzini's (1 993) 
thoughts on the knowledge and skills necessary for 
effective IR; they found that practitioners employed 
in the field for a greater number of years, those with 
doctoral degrees, those with the title of associate 
director, and those who reported directly to the 
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institution's president perceived themselves to 
be more effective. Storrar (1981) determined that 
institutional researchers at large, public universities 
experience role conflict that impinges upon their 
perceived effectiveness. 

The IR profession has benefited from turning 
its analytic lens back upon itself. Some clear 
patterns have emerged about how practitioners can 
negotiate professional challenges and increase their 
effectiveness, which the literature has operationally 
defined as having a tangible impact on decision- 
making, planning, and policy formation. Still, 
more of the story remains to be told. Many of the 
suggestions for improving effectiveness made by 
IR theorists and practitioners, while based upon 
valuable lived experience, were not arrived at 
through empirical research. Further the research 
studies that have been carried out to determine 
correlates of effectiveness have been limited by the 
fact that the dependent variable is self-reported 
effectiveness. While not wishing to impugn the 
importance of this work or the responses of our 
colleagues, it does seem that validation of self- 
reported effectiveness, through such means as 
feedback from colleagues (Delaney, 2001) would 
add substance to this line of inquiry. Finally, the 
studies carried out thus far have all been within 
the objectivist, deductive, positivist paradigms, 
which assume that truth exists independently of 
experience, simply waiting to be discovered and 
having the same meaning for all (Crotty,1 998). One 
of several alternative approaches to understanding 
IR effectiveness includes using a constructionist 
epistemology, a related theoretical perspective 
such as phenomenology, and methods such as 
interviews, document, analysis, and observation. 
Such an inductive approach holds that meaning 
is constructed by human beings as they engage 
with the world, that the possibility for new 
meaning emerges when we lay aside our prevailing 
understanding, and that depth and detail emerge 
when data collection and analysis are not limited to 
preexisting categories (Patton, 2002). The goal of 
this study is to use such an alternative approach to 
determine how institutional researchers who have 
been identified as effective by their colleagues view 
effectiveness in the profession. 



Method 

This study was carried out using qualitative 
research methods since the research questions are 
descriptive and open-ended in nature and require 
somewhat lengthy responses from a small group 
of persons with particular viewpoints (Bogdan 
& Biklen, 1 998; Merriam, 1 998). Techniques of 
naturalistic inquiry were employed, which affected 
sampling techniques, participant selection, research 
design, and data analysis (Erlandson, Harris, Skipper, 
& Allen, 1993; Lincoln & Guba, 1985; Miles & 
Huberman, 1994; Patton, 2002). 

The initial pool of effective institutional 
researchers was established based upon 
nomination by colleagues throughout the 
country. E-mail messages asking practitioners to 
nominate colleagues were sent via the listprocs of 
AIR and several of its regional and state affiliates. 
Nominations were also solicited at the AIR national 
conference. These efforts yielded 26 nominations. 
The researcher then narrowed the list of candidates 
to a smaller number (eight) that provided for 
maximum variability in terms of the candidates' 
institutions, job titles, longevity in IR, and personal 
characteristics. This was important in order to 
determine how such characteristics might relate to 
effectiveness. Candidates were then contacted and 
asked if they were willing to participate in the study; 
all agreed. 

Participants submitted copies of their resumes 
to the researcher and participated in individual 
on-site interviews (except for one interview that 
was carried out via telephone), which were tape- 
recorded and captured in written transcripts. The 
appropriateness of the questions was confirmed 
by a national panel of experts who provided 
feedback about both the interview questions 
as well as an overall proposal for the study. The 
researcher maintained a reflective journal to record 
observations made during the research process. 

The reflective journal, analysis of resumes, and 
analysis of interview transcripts served as methods 
of data triangulation of the results (Patton, 2002). 

Data analysis yielded two types of findings: 
detailed descriptions of each case, which were 
used to document uniqueness, and shared patterns 
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that emerged across cases (Patton, 2002). Data 
analysis involved breaking material into small 
units of observation, developing initial themes or 
categories within the findings, and considering 
alternative interpretations that either confirmed 
the initial themes or lead to the creation of new 
ones. The researcher attempted to bracket his 
knowledge and presuppositions so as not to taint 
the findings (Crotty, 1 998), but rather to focus on 
participants' perspectives (Bogdan & Biklen, 1998). 
Two peer debriefers tested themes and alternative 
conclusions by examining interview transcripts, 
field notes, and participant resumes and by 
suggesting theme independently of the researcher 
(Lincoln & Guba, 1985). Preliminary conclusions 
were shared with participants for their confirmation 
and elaboration; this constitutes a member check 
(Lincoln & Guba, 1 985). An audit trail of study 
materials will serve to provide for dependability and 
confirmability. All names noted in the results are 
pseudonyms. 

The researcher sought to confirm that the 
study's findings resulted from the participants' 
responses rather than his own preconceptions 
by situating his background and bracketing his 
assumptions. I am an IR professional with 1 6 
years' experience and considerable involvement in 
professional development activities. I am greatly 
influenced by Terenzini's (1 993) thoughts about IR 
effectiveness. I think that institutional researchers 
need to exhibit technical-analytical, issues, and 
contextual intelligence to be effective, but I also 
think that in other than one-person offices, the 
leader need not be cutting-edge on all three as long 
as the office as a whole is. I think the leader will be 
most effective if he or she is especially well versed 
in contextual and issues knowledge. I also feel very 
strongly that IR is only effective if it has a "seat at 
the table" in terms of access to management issues 
and decisions. I believe that effectiveness is highly 
contextual — what it takes to be effective in one 
institution at one time may be quite different from 
what it takes in other settings, or even in the same 
setting at a different point. It also seems reasonable 
to me that those with longer careers and those 
with greater tenure in a given setting will be more 



effective. I think those who have had greater access 
to and taken greater advantage of professional 
development opportunities will be more effective; I 
think involvement in AIR is critical here. 

Findings 

Profiles of the Participants: Elizabeth, Frank, Henry, 
Kim, Linda, Marshall, Martha, and Susan 

The analysis of resumes and office activities and 
characteristics may have served more to illustrate 
the diversity of the participants than to point to 
the common hallmarks of their effectiveness. Five 
were female and three were male. Seven were 
Caucasian and one was African American. Two 
were in their forties and six were in their fifties. 
Longevity in their current jobs ranged from 2 to 
31 years and averaged 16. Participants' average 
years in IR were 1 8 with a range from 2 to 34. Total 
years of employment in higher education ranged 
from 9 to 36 with an average of 25. While Elizabeth 
had a staff of only herself and an administrative 
assistant, Susan's staff was 15 FTE; average IR staff 
size was 6 FTE. Six of the participants worked in 
public institutions and two in private. Six worked 
in four-year institutions and two in two-year. Four 
of the participants had doctoral degrees and two 
more were pursuing doctorates. The disciplines of 
participants' highest degrees included Educational 
Leadership (2), Higher Education (2), Accounting, 
Computer Science, Political Science, and 
Psychology. Supervisors included two presidents, 
three chief academic officers, a senior vice 
president, an associate provost, and a vice president 
for information technology. Participants were 
involved in a variety of professional organizations as 
members and leaders, including AIR and its regional 
and state affiliates and special interest groups, 
EDUCAUSE, the Society for College and University 
Planning, accrediting bodies, and other higher 
education professional associations. Several have 
done consulting, and some have directed grants. 
Many had an impressive array of publications and 
presentations. Most had received professional 
awards and recognitions. 

Only two commonalities were evident from the 
analysis of resumes and from observations. First, 
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the magnitude and breadth of activities within all 
of the participants' offices was quite large. They did 
lots of work and lots of different work. Examples 
include coordination of university strategic planning 
and academic program review; administration of 
data warehouses and management information 
systems; response to internal and external data 
requests; state and federal reporting; administration 
of a broad program of surveys; coordination and 
consultation concerning assessment of student 
learning; research focusing upon enrollment 
management, financial issues, faculty workload, and 
faculty salaries; and coordination of accreditation 
activities. 

"We do it all"was a phrase used several times 
to characterize their activities. Some of the 
participants even had responsibilities not generally 
associated with IR, such as managing information 
technology, overseeing a testing center, and serving 
as the president's chief of staff. Interestingly, 
although there might be some relationship to 
institutional size, two- and three-person offices 
seemed to be engaged in as wide an array of 
activities as those with much larger staffs. The other 
commonality was a large degree of experience 
among the office staff. For example, although Kim's 
professional experience in IR and in her current 
role specifically was relatively low as compared 
to most of the other participants, she noted that 
her colleagues have over 70 years of combined 
experience. 

Themes 

Understanding your institution, multiple 
perspectives of its constituents, and academe 
in general. A theme expressed by nearly all of 
the participants was that effective institutional 
researchers understand how their institutions 
work (i.e., institutional politics, priorities, culture, 
personalities, decision-making processes). Henry 
referred to this as "figuring out how things get 
done at the institution." He observed that new 
institutional researchers have great technical skills 
but don't have an appreciation for context or 
understand why it is important to have it. Some of 
the participants noted that this knowledge is 



critical in senior IR positions, and that it comes from 
experience. 

A related idea is that it is important for 
institutional researchers to understand the multiple 
perspectives of different constituencies at their 
institutions. For example, the president, provost, 
chief financial officer, and faculty members may 
have very different perspectives on faculty salaries 
and may react differently to benchmarking with 
other institutions that is carried out by the IR 
office. Kim advised that institutional researchers 
"be flexible, recognize that multiple answers are 
possible." Marshall spoke of the need to understand 
how "information is used for decision-making, 
gathered, analyzed, conveyed, and considered." 

He also noted the importance of recognizing 
that "People don't always consider information in 
decision-making due to the press of time." Finally, 
he spoke of the importance of understanding how 
political dynamics change, for example, with the 
transition to a new president. 

Frank and Susan attribute their decisions to 
become experts in certain management processes 
in higher education as contributing to their 
effectiveness. Frank's involvement in environmental 
scanning "improved his ability to relate to academic 
departments, this improved [his] effectiveness 
and garnered a certain level of respect." Susan's 
involvement in total quality management 
"helped clarify [her] thinking about institutional 
research and higher education in general." These 
participants' knowledge of their institutional 
cultures led them to realize that gaining this 
expertise would make them more effective in their 
roles. 

Foundational to understanding one's own 
institution is understanding academe in general. 
Participants spoke of how the tradition of shared 
governance causes colleges and universities to act 
differently from for-profit or even other non-profit 
institutions when accountability and improvement 
information is considered. Frank says that he 
was helped by the fact that he is a "student of 
organizations." 

Engagement, visibility, relationships and trust. 
Related to understanding institutional culture is 
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being engaged within the institution, being visible, 
establishing and maintaining relationships, and 
building trust. This was the only theme to which 
all of the participants spoke. These actions are 
necessary for institutional researchers' results to be 
used effectively. Marshall noted that "Building good 
relationships is absolutely essential for people to 
take to heart what the data say." Frank attributed his 
effectiveness to the fact that "I wheedle my way into 
everything." Elizabeth offered the following: 

Gauging how effective you are is [related 
to] how many people know about you. 

And I can tell you that everybody on this 
campus knows who I am. Why is that? It's 
because what we provide ... is done in such 
a way that we end up serving everybody 
one way or another. .. . I am the type of 
person who is very outgoing. I know that 
some institutional research folks have the 
tendency to just sit in their offices and not 
interact too much, but the key is building 
relationships, and being out there, and being 
very responsive with quality stuff quickly. . . 

. I think effectiveness is related to being able 
to build relationships and gain trust, having 
your product on high demand and being 
used, and being able to make suggestions 
and being proactive rather than reactive, 
getting to know the operation of the college 
so well, and making suggestions in areas 
that people didn't think about. 

Linda said that to be effective as an institutional 
researcher, "you must have your office be a player, 
be respected, [and] have your office taken seriously." 
She said you must "have the connections to get 
your results used." These connections include 
both formal and informal contacts with colleagues 
throughout the institution. Such contacts included 
colleagues in Admissions (Frank), information 
technology (Linda), academic affairs (Martha), and 
in the faculty, supervisors (Linda), spouses who are 
employed at the institution (Linda), and various 
social contacts (Linda). Susan noted: 

Related to that is it depends upon the 
person receiving the information. I have 
worked for eight academic officers at least 



and I've worked from the extreme of 
"you do the analysis, but I want to get the 
data set". . . to the other extreme of 
"give me the bottom line." So it's sizing 
up your particular administrator and 
developing your responses according to 
what they best need and how far they 
want to dig down. 

Frank said that many of his best ideas have 
resulted from conversations during smoking breaks. 
Henry spoke about "finding] points of influence 
and chiseling away at them." He felt this is necessary 
for"getting the organization to be tough on itself." 
Elizabeth said "You get to be proactive by knowing 
people very well." She stated: 

Being part of the Cabinet makes me 
extremely effective because you know 
what is needed at the highest level and 
you understand what is expected. If you 
are at a lower level you may never know 
exactly what is needed. Reporting to the 
president is key and being part of Cabinet is 
tremendously helpful. It is also important to 
use this opportunity to contribute to show 
your value. 

Similarly, Susan felt that the relationships 
she has established have helped her to function 
effectively as an intermediary among leaders and to 
carry bad news forward successfully. Susan stated: 

A term I like to use a lot is fly below the radar. 
It's times I know I have moved information 
from one end of the administration building 
to the other ... in a non-threatening way. 

I know it sounds trite, but I work for the 
greater glory of [institution name]. I'm a 
[institution] alum, I care deeply about this 
place. 

Susan spoke about the opportunity to learn from 
mentors: 

I've definitely been blessed by fabulous 
mentors. . . Several of them have gone 
on too and are presidents of different 
universities. . . . They were all fabulous men. 
They included me from the very beginning 
in meetings and discussions. I had to be 
part of the conversation so I understood the 
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thinking process So now I try to include 

my staff in conversations wherever I can 
because you have to be in the conversation 
to get how people think about things, to 
understand what the other tangential issues 
are that are not always easy to identify. 

Participants also spoke about lack of 
engagement and trust and problems with 
relationships and their consequences for 
effectiveness in IR. Kim explained that institutional 
researchers should understand that people get 
defensive when IR is viewed as an interloper. 

Martha discussed getting caught in the middle 
between strong personalities. Linda's effectiveness 
was hampered by a prior provost's tendency 
towards secrecy and inclination to "shoot the 
messenger." Frank discussed the difficulties of 
working for a leader who didn't believe in using 
information to make decisions. Martha's problems 
with people asking her to do things that are beyond 
her capability and "offices not following standard 
reporting methodologies so that they look better" 
could be attributed to poor relationships and lack 
of trust. Henry noted that a barrier to having his 
office's work be used to a greater extent lay in the 
fact that the institution perceived itself as very 
successful and viewed IR as overly critical: 

I really think that successful institutions are 
the ones that have the hardest job making 
a change. This place has never been in a 
crisis. . . . We got 1 0 years of re-accreditation 
with a totally clean slate, we passed our levy 
with a (large) affirmative vote, and we were 
re-validated as a member of the League for 
Innovation in the Community College. Well, 
then, the [state] Performance Report came 
out and then they were like "what's IR trying 
to do here, throwing all this mud on our 
faces." 

Other practitioners discussed the problems 
associated with IR having poor visibility on 
campus. Susan noted that institutional researchers 
need to be included in key discussions in order 
to be effective and recounted problems that she 
experienced when the physical location of the 
office was changed. The old adage out of sight, out 



of mind came into play, and she found herself less 
likely to be included in important dialogues. She 
responded by finding ways to get herself and her 
colleagues into other locations frequently. She 
maintains a small secondary office in the campus' 
main administration building. 

Susan also commented upon the fragility and 
precariousness of relationships that institutional 
researchers establish with others at their 
institutions. Trust is easily lost, and the work of 
building relationships is never finished. She posited 
the notion that it often takes considerable time for 
IR's benefit to be realized: 

I tell new staff when they come on board 
that you will see the impact of your work, 
but it may take a couple of years. The 
gestation period is quite lengthy. But if 
you stick around long enough, you'll see 
a particular analysis that you know has 
implications on how the University ought 
to think about creating new programs, 
eliminating new programs, or just helping 
them to chart their course. You'll see it. And 
that is probably what is most satisfying. I 
do think that for me that is the gauge of 
effectiveness. 

She said that"IR is in a tremendous position to 
influence, but has no real power," and offered: 

The other thing that I always tell new staff 
is that we exist only because someone 
finds our work valuable. We are not paying 
people. We are not registering students. We 
are not paying the bills. We are not cleaning 
the offices. Our analytical work must be of 
a measure that people find valuable, or we 
don't exist. 

Finally, she volunteered that the work necessary 
to develop relationships can lead to overload, 
therefore institutional researchers have to balance 
their interactions and other work. 

Helping leaders make meaning ofIR findings and 
use ofIR work in the institution. A related but distinct 
theme was that successful institutional researchers, 
as Henry said, "do not just present numbers," but 
actively work with leaders to understand their 
relevance to institutional operations and planning. 
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of the barriers he has had to overcome in order to 
be effective is "lack of user sophistication . . . [you] 
first need to [have] the necessity to use information, 
then get decision-makers to understand how to 
take action based on information." He felt strongly 
that "IR needs to take an educational approach," 
and offered that he does monthly seminars for 
department chairs to help them understand IR 
information and use it more effectively. He said 
that "with changes in technology, IR's role has 
shifted from providing information to educating 
users about it." Similarly, Susan said that the 
successful institutional researcher knows howto 
"size up what is pertinent and [not] bury people in 
data." She offered that she "does more advising to 
senior administration than anything else." Related 
to the earlier themes, Elizabeth stated that lack 
of utilization of IR information is most effectively 
overcome by"being out there and knowing the 
context, understanding what is useful." 

Viewing IR as a means to a greater goal. Perhaps 
as a consequence of understanding context, 
becoming engaged, and developing trust, several 
of the participants spoke about the end goal of 
successful IR as not having one's information used 
but rather helping to make one's institution better. 
Martha, Frank, and Linda felt that helping students 
to succeed is an end product of IR that is particularly 
important and satisfying to them. Frank stated: 
Anybody can report anything effectively. 
Anybody can fill out IPEDS forms. What you 
need to ask yourself is if what you do makes 
a difference in the lives of the students and 
the campus. If you can answer yes to that, 
you are being effective. I'd die if all I did was 
fill out IPEDS forms, I'd literally die. 

He said that IR is critical in helping the institution to 
answer the question "are we getting better?" 

Professional colleagues and professional 
development. While developing relationships with 
people at one's institution was critically important 
for the participants, they also spoke to the 
importance of developing a network of colleagues 
outside of one's institution. The Association for 
Institutional Research was often noted as an 
important resource for developing a professional 



network, as were state and regional AIR affiliates 
and data exchange groups. Linda said that this 
helps her keep aware of best practices, and Kim 
offered that this helps her and her office staff with 
developing specific skill sets. Martha spoke of 
"sharing what I do with others to get feedback." 
Frank tries to travel as much as he can to present 
and consult and noted that "the true benefit 
of consulting is seeing what others are doing." 

Henry noted that "good support for professional 
development" is extremely important, and Elizabeth 
valued having a budget to spend as she sees fit to 
promote professional development. 

Workload, time management, and tools. Several 
of the participants discussed workload as a barrier 
to effectiveness and successful time management 
as a tool to overcome it. Elizabeth spoke about 
the importance of "learning how to say no when 
necessary" and Linda discussed being "judicious 
about committee memberships." Linda also talked 
about developing the discipline to force oneself to 
"get out of the office and focus on critical issues." 
Several participants spoke of finding what Martha 
termed "time to read and think." Kim said that an 
important strategy for effectiveness is "leveraging 
the tools you have available to get things done as 
effectively as possible" and noted that she is "always 
going after new technology" to help her do this. 

Being self-reflective and proactive. The 
experiences of the participants suggest that the 
institutional researcher who understands the 
context of his or her work, is engaged with his or 
her institutional and professional communities, 
and who has successfully managed workload 
will be self-reflective and proactive. Most of 
the participants spoke about the importance 
of working at effectiveness every day, actively 
listening, being proactive, being able and willing to 
learn new things, and constantly analyzing IR's role 
in the institution. Linda noted that the effective 
institutional researcher understands the importance 
of and has the ability to constantly "translate 
campus discussions into research questions." 
Elizabeth noted the importance of doing systematic 
IR program reviews for facilitating self-reflection. 
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Attention to detail and quality control. A theme 
expressed by several of the participants that is 
related to self-reflection but nevertheless distinct 
is the need for institutional researchers to show 
attention to detail and exercise quality control. 
Martha discussed the need to "have a critical eye 
for data consistency and integrity." Henry felt 
strongly that "IR must have the final quality check," 
discussing this in the context of the institutional 
perception that "anything that comes out of the 
data warehouse is accurate." Susan stated: 

Your work needs to be reliable, be credible, 
you need to be consistent. Obviously we like 
to do things flawlessly, but that realistically 
isn't going to happen, but you need to 
minimize errors because once you have sent 
out a data set or an analysis you don't want 
to come back two days later [with problems]. 
So we have a lot of processes in place to try 
to minimize that, but sometimes things still 
do happen. You have to know enough to say 
"this doesn't pass the test of reasonableness." 

Personal and professional qualities. Most of those 
interviewed listed key personal characteristics of 
effectiveness that might be considered traits of 
professionalism. These included being objective, 
creative, flexible, timely, accurate, logical, 
cooperative, and responsive; having a broad 
perspective; not sacrificing principles or ethical 
standards; being able to function under pressure; 
actively listening; knowing your own capabilities 
and biases; and being willing to embrace change. 
Several people also noted the importance of having 
a sense of humor; for example, Martha cited the 
need to smile when asked the same question 
the third time because the person has lost the 
information that she gave them before. 

Technical competency. Only a few of the 
participants noted the importance of technical/ 
analytical skills. Martha and Kim discussed the 
need to be able to work with various types of 
computer software; Elizabeth volunteered the 
importance of accessing, manipulating, and 
analyzing data without the support of information 
technology colleagues outside of the office; 
and Kim commented upon understanding the 



epistemological bases of research approaches 
and their corresponding methodologies. Several 
persons noted that not everyone in a multi-person 
IR office needs to have the full complement of 
technical/analytical skills as long as they are found 
overall among the staff. 

Discussion 

The participants provided information about 
effectiveness in IR and how that effectiveness, 
which may be similar to some of the existing 
literature, can be improved to be even richer 
and more nuanced. A heuristic model of the ten 
themes related to effectiveness is shown in Figure 
1. Some of the themes could be clustered. For 
example, engagement, visibility, relationships and 
trust; helping leaders make meaning of IR findings 
and use of IR work in the institution; and viewing 
IR as a means to a greater goal might share the 
underlying concept of institutional engagement. 
Professional colleagues and professional 
development; workload, time management and 
tools; and being self-reflective and proactive might 
cluster around the common concept of planning 
and improvement. Finally, attention to detail and 
quality control; personal and professional qualities; 
and technical competency might be collectively 
construed as a personal and professional dimension 
of effectiveness in IR. 

The results confirmed Terenzini's (1 993) 
contention that contextual knowledge and skills 
(e.g., understanding of the institutional culture, 
history, politics, personalities) are critical for success 
in IR. Developing this set of knowledge and skills 
and cultivating relationships allows institutional 
researchers to establish and maintain trust. 
Important technical/analytical skills (Terenzini, 

1993) were also noted, such as being able to 
work with various types of computer software 
and accessing, manipulating, and analyzing data 
independently. 

As Delaney (2001) found, workload and lack of 
recognition served as barriers to effectiveness in 
IR. Other barriers included lack of utilization of the 
products and services provided by the IR office, 
lack of user sophistication, lack of data quality 
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from cooperating offices, and the presence of the 
IR office in a low visibility location on campus. 
Opportunities that effective IR practitioners took 
advantage of and supports that they proactively 
developed included access to resources for 
professional development, attendance at 
conferences, visits to other campuses, developing a 
strong professional network (also noted by Delaney, 
2000), access to institutional leaders (Huntington 
& Clagett, 1 991 ), mentoring (Delaney, 2001 ), 
becoming involved as members and leaders in 
campus groups and in professional organizations, 
and becoming experts in areas of specialization. 

While the results of this study are not 
intended to generalize to all institutional 
researchers or even to all those deemed particularly 
effective, they nevertheless provide some 
implications for practitioners and for those who 
impact their professional preparation. Just as 
many years of research about the effect of college 
on students has clearly determined that "What 
students do during college counts more in terms 
of desired outcomes than who they are or even 
where they go to college" (Kuh, 2001, p. 1), this 
study suggests that what institutional researchers 
do in their jobs is more important than their 



backgrounds, institutional settings, and prescribed 
tasks. Effective institutional researchers develop a 
keen understanding of people and processes and 
use this understanding to tailor their activities and 
disseminate them effectively (Augustine, 2001). 

They are involved in an abundance of activities, 
interact with a diverse array of people, and cultivate 
the variety of professional characteristics listed 
above. They overcome barriers by taking advantage 
of opportunities provided to them and proactively 
cultivating others. Strategies for those facilitating 
the preparation of institutional researchers 
include articulating the characteristics of effective 
IR, pairing aspiring and new professionals with 
effective practitioners early and often, and assisting 
them with developing an ongoing capacity to 
gauge their own effectiveness. 

Further research might involve quantitatively 
analyzing structural relationships between 
components of effectiveness in IR as measured by a 
survey. One of several potential structural models 
that could be evaluated is shown in Figure 2. A 
large-scale data collection could test the validity of 
the model posed here and ultimately provide useful 
information about employee selection, retention, 
and professional development needs in IR. 
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Figure 1. Heuristic model of effectiveness in institutional research. 
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Figure 2. structural model of effectiveness in institutional research. 
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